Claims: I claim: 

1 

1 1, A method for enhancing image quality comprising: 

2 developing a usefulness metric which identifies a 

3 limit to sharpness enhancement that can be applied to 

4 decoded video without enhancing coding artifacts; and 

5 applying the usefulness metric to at least one 

6 sharpness enhancement algorithm, the usefulness metric and 

7 the sharpness enhancement algorithm being separate such 

8 that the usefulness metric can be used with a variety of 

9 algorithms. 
1 

1 2. A method for enhancing the sharpness of a coded digital 

2 video, comprising the steps of: 

3 selecting and extracting statistical information from 

4 a coded video bit stream in order to identify the video's 

5 coding complexity; 

6 based upon the coding complexity, developing a 

7 usefulness metric for the coded video, which identifies a 

8 limit to sharpness enhancement that can be applied to the 

9 coded video after it is decoded, without enhancing coding 
10 artifacts; and 



US010005APP 



Page 15 of 29 



1U ^applying a sharpness enhancement algorithm to the 

12 decoded video to increase sharpness within the limit 

13 prescribed by the usefulness metric. 
1 

1 3, The method as claimed in claim 2 wherein the sharpness 

2 enhancement algorithm is a peaking algorithm 
1 

1 4 . The method as claimed in claim 2 wherein the sharpness 

2 enhancement algorithm is a spatial-domain algorithm 
1 

1 5. The method as claimed in claim 2 wherein the usefulness 

2 metric is calculated on a pixel-by-pixel basis 
1 

1 6. The method as claimed in claim 2 wherein the coding 

2 complexity is defined as the product of a quantization 

3 parameter and a number of bits used to code a macro block 
l 

1 7. The method as claimed in claim 2 wherein the coding 

2 complexity is defined as the product of a quantization 

3 parameter and a number of bits used to code a block. 
1 

1 8. The method as claimed in claim 2, wherein the usefulness 

2 metric occupies a range, a first terminus of the range 

3 . meaning no sharpness enhancement is allowed for a 
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4, particular pixel and second terminus of the range meaning 

5 that the pixel can be freely enhanced 

1 

1 9. The method as claimed in claim 2, wherein the method is 

2 also applied to skipped macroblocks, the usefulness metric 

3 being estimated based upon the coding complexity of 

4 surrounding macro blocks or the coding complexity of a 

5 previous frame 
1 

1 10. The method as claimed in claim 2, wherein the method is 

2 also applied to uncoded blocks, the usefulness metric being 

3 estimated based upon the coding complexity of surrounding 

4 blocks or the coding complexity of a previous frame 
1 

1 11. The method as claimed in claim 2, wherein in addition 

2 to the usefulness metric, scene-content related information 

3 is incorporated into a coding gain calculation 
1 

1 12. The method as claimed in claim 2, wherein the scene- 

2 content related information is derived from edge 

3 informaticpn 
1 

1 13. The method as claimed in claim 5, wherein coding gain 

2 of a pixel is determined by the equation: 
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3. 

4 * g C oding(i,j) = UME(i f j) + gedge(i.j) 

5 

6 

7 and wherein i and j are pixel coordinates, g C oding is the 

8 pixel coding gain, UME is the usefulness metric and g e dge is 

9 based upon edge-related information derived from the image 
1 

1 14. The method as claimed in claim 13, wherein spatial low™ 

2 pass filtering is applied to a complexity map calculated 

3 from the coded digital video. 
1 

1 15. The method as claimed in claim 13, wherein temporal 

2 filtering is applied to the coding gain using the coding 

3 gain of a previous frame 
1 

1 16. The method as claimed in claim 13, wherein the equation 

2 can be extended to include an additional term directly 

3 related to the quantization parameter 
1 

1 17. The method as claimed in claim 6, wherein a block-based 

2 complexity map is filtered temporally using an IIR filter. 
1 

1 18. The method as claimed in claim 6, wherein a macro 

2 block-based complexity map is filtered temporally using an 

3 IIR filter. 
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U 19. TJne method as claimed in claim 17 or 18, wherein the 

2 temporal filtering is in accordance with the following 

3 equation: 
4 

5 compl MB/biock (r,s,t) = k * compl M B/biock (r,s,t) + seal * (1- 

6 k) * compl MB/biock (r,s,t-l) 
7 

8 and wherein r,s is the spatial coordinate of a macro block 

9 or block, t represents the current picture, k is the IIR 

10 filter coefficient and seal is a scaling term taking into 

11 account picture complexity determined by the image's 

12 picture type. 
1 

1 20. A device for image quality enhancement comprising: 

2 a peaking filter which filters a decoded luminance 

3 signal, generating a high pass signal; 

4 a plurality of pixel based control blocks, operating 

5 in parallel on the decoded luminance signal, each 

6 calculating a maximum allowable gain factor, based upon a 

7 characteristic of the luminance signal, wherein at least 

8 one control block is a coding gain block which implements a 

9 usefulness metric which determines the allowable amount of 

10 peaking; 
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11. dynamic gain control for selecting a minimum gain 

12 based upon the calculated maximum gain factors; 

13 a multiplier for multiplying the high pass signal by 

14 the minimum gain generating a multiplied signal; and 

15 an adder for combining the decoded luminance signal 

16 with the multiplied signal, generating an enhanced signal. 
1 

1 21. A device as claimed in claim 22, wherein the control 

2 blocks comprise: 

3 a contrast control block; 

4 a dynamic range control block; 

5 a clipping prevention control block; 

6 an adaptive coring control block; and 

7 a coding gain block, all of the blocks being connected 

8 in parallel 
1 

1 22. A device for enhancing the image quality of a digital 

2 video comprising: 

3 a usefulness metric generator which identifies a limit 

4 to sharpness enhancement that can be applied, without 

5 enhancing coding artifacts, to decoded digital video; 

6 a controller which applies the usefulness metric to at 

7 least one sharpness enhancement algorithm, the usefulness 

8 metric and the sharpness enhancement algorithm being 
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9 + separate such that the usefulness metric can be used with a 

10 variety of algorithms. 
1 

1 23. A system which enhances sharpness of a coded digital 

2 video f comprising: 

3 a selector which selects and extracts statistical 

4 information from a coded video bit stream in order to 

5 identify the video's coding complexity; 

6 a usefulness metric generator that, based upon the 

7 coding complexity, develops a usefulness metric for the 
y0 8 coded digital video after decoding, which identifies a 

§1 9 limit to sharpness enhancement that can be applied to a 

-P 10 decoded video without enhancing coding artifacts; and 
~ 3 11 a sharpness enhancer which applies a sharpness 

H 12 enhancement algorithm to the decoded video to increase 

n 13 sharpness within the limit prescribed by the usefulness 

14 metric. 
1 

1 24. The system as claimed in claim 23, wherein the 

2 sharpness enhancement algorithm is a peaking algorithm. 
1 

1 25. The system as claimed in claim 23, wherein the 

2 sharpness enhancement algorithm is a spatial-domain 

3 algorithm. 
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1, 26. The system as claimed in claim 23, wherein the 

2 usefulness metric is calculated on a pixel-by-pixel basis. 

1 

1 27. The system as claimed in claim 23, wherein the coding 

2 complexity is defined as the product of a quantization 

3 parameter and a number of bits used to code a macro block 
1 

1 28. The system as claimed in claim 23, wherein the coding 

2 complexity is defined as the product of a quantization 

3 parameter and a number of bits used to code a block 
1 

1 29. The system as claimed in claim 23, wherein the 

2 usefulness metric occupies a range, a first terminus of the 

3 range zero meaning no sharpness enhancement is allowed for 

4 a particular pixel and a second terminus of the range 

5 meaning that the pixel can be freely enhanced. 
1 

1 30. The system as claimed in claim 23 wherein the system is 

2 also applied to skipped macro blocks, the usefulness metric 

3 being estimated based upon the coding complexity of 

4 surrounding macro blocks or based upon the coding 

5 complexity of a previous frame. 
1 
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U 31. The system as claimed in claim 23, wherein the system 

2 is also applied to uncoded blocks, the usefulness metric 

3 being estimated based upon the coding complexity of 

4 surrounding blocks or the coding complexity of a previous 

5 frame. 
1 

1 32. The system as claimed in claim 23, wherein in addition 

2 to the usefulness metric, scene-content related information 

3 is incorporated into a coding gain calculation. 

O 1 

5j 1 33. The system, as claimed in claim 32, wherein the scene- 

ff 1 2 content related information is derived from edge 

J: 3 information. 

U 1 

U 1 34. The system as claimed in claim 23, wherein coding gain 

D 2 of a pixel is determined by the equation: 
3 

4 gcoding(i.j) = UME(i,j) + gedge(irj) 
5 

6 and wherein i and j are pixel coordinates, g C oding is the 

7 pixel coding gain, UME is the usefulness metric and g e d ge is 

8 derived from edge-related information derived from the 

9 image . 
1 



US010005APP 



Page 23 of 29 



U 35. The system as claimed in claim 23, wherein spatial 

2 low-pass filtering is applied to a complexity map 

3 calculated from the coded digital video stream. 
1 

1 36. The system as claimed in claim 23, wherein temporal 

2 filtering is applied to a coding gain based upon a coding 

3 gain of a previous frame. 
1 

1 37. The system as claimed in claim 34, wherein the 

p 2 equation can be extended to include an additional term 

5 3 directly related to the quantization parameter. 
CP 1 

£ 1 38. The system as claimed in claim 36, wherein a block- 

;\ 2 based complexity map is filtered temporally using an IIR 

2 3 filter. 

1 39. The method as claimed in claim 36, wherein a macro 

2 block-based complexity map is filtered temporally using an 

3 IIR filter. 
1 

1 40. The system as claimed in claim 36, wherein the 

2 temporal filtering is- in accordance with the following 

3 equation: 
4 
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5. compL MB/biock (r,s,t) = k * compl M B/biock (r,s,t) + seal * (1- 

6 k) * compl MB/biock (r,s,t-l) 

7 

8 and wherein r,s is the spatial coordinate of a macroblock 

9 (MB) or block, t represents the current picture, k is the 

10 IIR filter coefficient and seal is a scaling term taking 

11 into account picture complexity determined by the image's 

12 picture type. 
1 

1 41. Computer-executable process steps to enhance image 

2 quality, the computer-executable process steps being stored 

3 on a computer-readable medium and comprising: 

4 an extracting step to extract statistical information 

5 from a coded video bit stream in order to identify a 

6 video's coding complexity; 

7 a generating step to generate a usefulness metric for 

8 a coded video based upon the coding complexity, which 

9 identifies a limit to sharpness enhancement that can be 

10 applied to the coded video after decoding without enhancing 

11 coding artifacts; and 

12 an enhancement step to enhance the sharpness of the 

13 image by applying a sharpness enhancement algorithm to a 

14 decoded video to increase sharpness within the limit 

15 prescribed by the usefulness metric. 
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L 

1 42. Means for enhancing the sharpness of a coded digital 

2 video, comprising: 

3 extracting means for extracting statistical 

4 information from a coded video bit stream in order to 

5 identify the coded digital video's coding complexity; 

6 generating means for developing a usefulness metric 

7 for the coded digital video, based upon the coding 

8 complexity, which identifies a limit to sharpness 

9 enhancement that can be applied to the coded digital video 

10 after decoding without enhancing coding artifacts; and 

11 enhancement means for applying a sharpness enhancement 

12 algorithm to a decoded video to increase sharpness within 

13 the limit prescribed by the usefulness metric. 
1 

1 43. A signal, embodied in a carrier wave, representing data 

2 for enhancing sharpness of a decoded digital video, 

3 comprising: 

4 statistical information selected from a coded video 

5 bit stream to be used in identifying the complexity of a 

6 video; 

7 a usefulness metric, based upon the complexity of the 

8 video, which identifies a limit to sharpness enhancement 



USO 10005 APP 



Page 26 of 29 



9. which can be applied to the decoded video without enhancing 

10 coding artifacts; and 

11 a sharpness enhancement algorithm to be used for 

12 increasing the sharpness of the decoded video within the 

13 limit prescribed by the usefulness metric. 



1 

1 44. A method for enhancing image quality comprising the 

2 steps of: 

3 peaking filtering a coded luminance signal, increasing 

4 the amplitude of the luminance signal and generating a high 

5 pass signal; 

6 calculating at least one maximum gain factor for the 

7 luminance signal, based on a characteristic of the 

8 luminance signal, wherein at least one gain factor 

9 calculation implements a usefulness metric which determines 

10 an allowable amount of peaking which will not intensify 

11 coding artifacts; 

12 selecting a minimum gain from the maximum gain 

13 factors; 

14 multiplying the high pass signal by the minimum gain 

15 generating a multiplied signal; and 

16 adding a decoded luminance signal with the multiplied 

17 signal, generating an enhanced signal. 
1 
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L 45. *A video receiving device comprising: 

2 a peaking filter which filters a decoded luminance 

3 signal, generating a high pass signal; 

4 a plurality of pixel based control blocks, operating 

5 in parallel on the decoded luminance signal, each 

6 calculating a maximum allowable gain factor, based upon a 

7 characteristic of the luminance signal, wherein at least 

8 one control block is a coding gain block which implements a 

9 usefulness metric which determines the allowable amount of 

10 peaking; 

11 a dynamic gain control for selecting a minimum gain 

12 based upon the calculated maximum gain factors ; 

13 a multiplier for multiplying the high pass signal by 

14 the minimum gain generating a multiplied signal; and 

15 an adder for combining the decoded luminance signal with 

16 the multiplied signal, generating an enhanced signal. 
17 
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